Papers with analogical reasoning
Analogical Reasoning on Chinese Morphological and Semantic Relations (P18-2)
Copied to clipboard
| Challenge: | Analogical reasoning is effective in capturing linguistic regularities. |
| Approach: | They propose to use Chinese lexical knowledge to build an analogical reasoning task using a large dataset. |
| Outcome: | The proposed dataset proves to be reliable benchmark for evaluating Chinese word embeddings. |
Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research (2026.tacl-1)
Copied to clipboard
| Challenge: | Analogical reasoning is an essential aspect of human cognition, says aaron eliotta . eelisa e. sabet: some have argued that analogy is central to the human cognitive experience . |
| Approach: | They summarize key theories about the processes underlying analogical reasoning from the cognitive science literature and relate it to current research in natural language processing. |
| Outcome: | The proposed approaches are relevant for several major challenges in natural language processing, not directly related to analogy solving. |
AnaScore: Understanding Semantic Parallelism in Proportional Analogies (2025.naacl-long)
Copied to clipboard
| Challenge: | AnaScore metric aims to evaluate the strength of semantic parallelism in sentence analogies. |
| Approach: | They propose an automatic metric to evaluate the strength of semantic parallelism in sentence analogies. |
| Outcome: | The proposed metric shows that formally explainable examples are more beneficial for analogical reasoning, whereas ambiguous analogies with no clear criterion tend to hinder inference. |
ALERT: Adapt Language Models to Reasoning Tasks (2023.acl-long)
Copied to clipboard
Ping Yu, Tianlu Wang, Olga Golovneva, Badr AlKhamissi, Siddharth Verma, Zhijing Jin, Gargi Ghosh, Mona Diab, Asli Celikyilmaz
| Challenge: | Large language models have shown increasing in-context learning capabilities with scaling up the model and data sizes. |
| Approach: | They propose a benchmark and suite of analyses to evaluate reasoning skills of large language models. |
| Outcome: | The proposed model compares pre-trained and fine-tuned models on tasks that require reasoning skills to solve. |
In-Context Analogical Reasoning with Pre-Trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Analogical reasoning is a fundamental capacity of human cognition that allows us to reason abstractly about novel situations by relating them to past experiences. |
| Approach: | They apply large pre-trained language models to visual Raven’s Progressive Matrices (RPM) and use language-based abstractions to support analogy in AI systems. |
| Outcome: | The proposed language-based abstractions outperform human models on Raven’s Progressive Matrices and supervised vision-based methods. |
A Systematic Investigation of KB-Text Embedding Alignment at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge bases (KBs) and text often contain complementary knowledge. |
| Approach: | They propose a framework for aligning KB and text embeddings for joint reasoning . they also evaluate alignment methods to infuse textual information into KB embeddables . |
| Outcome: | The proposed framework can be used to predict link prediction on emerging entities and events using textual information. |
A Neural-Symbolic Approach to Natural Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have enabled deep neural networks to perform natural language understanding tasks, but their performance can drastically deteriorate when logical reasoning is needed. |
| Approach: | They propose a framework for NLU based on analogical reasoning based upon neural processing and logical reasoning using both neural and symbolic processing. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two NLU tasks, question answering (QA) and natural language inference (NLI). |
Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on word analogies, but they neglect structures that underpin analogical reasoning. |
| Approach: | They propose a task to abduct structures that form an analogy between two systems to evaluate their analogical reasoning abilities. |
| Outcome: | The proposed task is based on 400 scientific analogies from 13 different fields and is compared with a standard SCAR benchmark. |
Retrieval, Analogy, and Composition: A framework for Compositional Generalization in Image Captioning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches fail to generalize well to concepts that are not observed during training. |
| Approach: | They propose a framework that revolves around probing several similar image caption training instances and performing analogical reasoning over relevant entities in retrieved prototypes. |
| Outcome: | The proposed framework improves on the widely used image captioning benchmarks and on composition-related evaluation metrics. |
Metaphor Understanding Challenge Dataset for LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Metaphor understanding is an essential task for large language models (LLMs). |
| Approach: | They propose to evaluate the metaphor understanding capabilities of large language models (LLMs) the metaphor understanding challenge dataset provides over 10k paraphrases and 1.5k instances of inapt paraphrase. |
| Outcome: | The metaphor understanding challenge dataset evaluates the performance of large language models on a range of NLU tasks. |
Why does PairDiff work? - A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection (C18-1)
Copied to clipboard
| Challenge: | Existing methods for representing semantic relations between words are unclear . identifying relations between word and entity is important for NLP applications . |
| Approach: | They propose to compute the vector offset between word embeddings to represent relation between two words . they show that PairDiff is an uncorrelated bilinear operator that can be simplified to a linear form . |
| Outcome: | The proposed method is surprisingly accurate and can be used on multiple word embeddings. |
AnRe: Analogical Replay for Temporal Knowledge Graph Forecasting (2025.acl-long)
Copied to clipboard
| Challenge: | Temporal Knowledge Graphs (TKGs) are vital for event prediction, yet current methods face limitations. |
| Approach: | They propose a training-free Analogical Replay reasoning framework that uses LLMs to extract historical contexts and generate analogical reasoning examples as contextual inputs. |
| Outcome: | The proposed model outperforms existing training-free methods on four benchmarks. |
DeFine: Decision-Making with Analogical Reasoning over Factor Profiles (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models are ideal for decision-making, but they can be difficult to process when they are verbose and include repetition, hedging, and vagueness. |
| Approach: | They propose a framework that constructs probabilistic factor profiles from complex scenarios and integrates them with analogical reasoning to guide LLMs in making decisions in new situations. |
| Outcome: | The proposed framework separates the tasks of quantifying uncertainty and incorporating it into LLM decision-making. |
ePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Large language models have shown exciting progress on several NLP benchmarks . however, evaluating their ability for complex analogical reasoning remains under-explored . |
| Approach: | They propose a dataset of narratives for employing proverbs in context as a benchmark for abstract language understanding. |
| Outcome: | The proposed dataset provides fine-grained annotation of aligned spans between proverbs and narratives and contains minimal overlaps between narratives with proverb . the results show that large language models struggle on these tasks compared to humans, and these tasks pose multiple learning challenges. |
Investigating the Shortcomings of LLMs in Step-by-Step Legal Reasoning (2025.findings-naacl)
Copied to clipboard
Venkatesh Mishra, Bimsara Pathiraja, Mihir Parmar, Sat Chidananda, Jayanth Srinivasa, Gaowen Liu, Ali Payani, Chitta Baral
| Challenge: | Reasoning abilities of LLMs have been a key focus in recent years. |
| Approach: | They propose to use a college-level Multiple Choice Question-Answering task to identify LLM errors and evaluate their performance. |
| Outcome: | The proposed framework can be used in detailed error analysis of reasoning chains for logic-intensive complex tasks. |
AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies (2024.emnlp-main)
Copied to clipboard
Xiao Ye, Andrew Wang, Jacob Choi, Yining Lu, Shreya Sharma, Lingfeng Shen, Vijay Murari Tiyyala, Nicholas Andrews, Daniel Khashabi
| Challenge: | Analogical reasoning is an important part of human communication, says a new study . a benchmark to determine analogical reasoning ability in language models is needed . |
| Approach: | They propose to benchmark analogical reasoning ability in language models by collecting 340 analogies from human writings. |
| Outcome: | The proposed benchmark aims to determine analogical reasoning ability in language models. |
Concept-aware Data Construction Improves In-context Learning of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work curating in-context learners assumes that ICL emerges from vast over-parametrization or the scale of multitask training. |
| Approach: | They propose a framework for constructing training scenarios that make it beneficial for the LM to learn to utilize the analogical reasoning concepts from demonstrations. |
| Outcome: | The proposed framework makes it beneficial for the LM to learn to utilize the analogical reasoning concepts from demonstrations and fares comparably to previous in-context learners trained in large-scale multitask learning requiring magnitudes of more training data. |
Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing zero-shot LLM-based Vision-and-Language Navigation agents either encode images as textual scene descriptions, potentially oversimplifying visual details, or process raw image inputs, which can fail to capture abstract semantics required for high-level reasoning. |
| Approach: | They propose to integrate large language models into embodied AI models by incorporating textual descriptions that facilitate analogical reasoning across images from multiple perspectives. |
| Outcome: | The proposed approach improves the agent’s contextual understanding on the R2R dataset, showing that it can make better decisions based on the LLMs. |
Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performance (2023.emnlp-main)
Copied to clipboard
| Challenge: | Analogical reasoning is a common way to evaluate word embeddings in NLP, but it is also of interest to investigate whether or not it is able to be learned. |
| Approach: | They propose to use proportional analogies to evaluate word embeddings in NLP . they also test whether analogical reasoning is a task in itself that can be learned . |
| Outcome: | The proposed models can learn analogical reasoning even with small amounts of data. |
Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? (2025.findings-acl)
Copied to clipboard
| Challenge: | Analogical reasoning is a unique ability of humans to address unfamiliar challenges by transferring strategies from relevant past experiences. |
| Approach: | They propose to use self-generated random examples to improve performance on a variety of reasoning tasks by incorporating relevant examples from relevant past experiences. |
| Outcome: | The proposed methods achieve comparable or even better performance on GSM8K with random biological examples. |